{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro to LLMstudio\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import clients\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llmstudio import LLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can set OPENAI_API_KEY and ANTHROPIC_API_KEY on .env file\n",
    "\n",
    "claude2 = LLM(\"anthropic/claude-2.1\") # or you can pass api_key as argument here\n",
    "gpt4 = LLM(\"openai/gpt-4-1106-preview\")\n",
    "gpt3 = LLM(\"openai/gpt-3.5-turbo\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chat (non-stream)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'id': 'c0bd54fc-238f-401e-acdc-4992594c87e9',\n",
       " 'chat_input': \"What's your name\",\n",
       " 'chat_output': \"I am an AI developed by OpenAI, and I don't have a personal name like a human does. You can refer to me as ChatGPT, OpenAI's language model, or simply AI. How can I assist you today?\",\n",
       " 'timestamp': 1705588623.963223,\n",
       " 'provider': 'openai',\n",
       " 'model': 'gpt-4-1106-preview',\n",
       " 'metrics': {'input_tokens': 4,\n",
       "  'output_tokens': 49,\n",
       "  'total_tokens': 53,\n",
       "  'cost': 0.00151,\n",
       "  'latency': 3.4426088333129883,\n",
       "  'time_to_first_token': 1.334681749343872,\n",
       "  'inter_token_latency': 0.04298696226003219,\n",
       "  'tokens_per_second': 14.523869083285478},\n",
       " 'parameters': {'temperature': 1,\n",
       "  'max_tokens': 256,\n",
       "  'top_p': 1,\n",
       "  'frequency_penalty': 0,\n",
       "  'presence_penalty': 0}}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gpt4.chat(\"What's your name\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Async version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'id': 'a1c2004a-fb10-4847-86fe-ca18ff3e57e6',\n",
       " 'chat_input': \"What's your name\",\n",
       " 'chat_output': \"I'm an AI language model created by OpenAI, known as ChatGPT. I don't have a personal name since I'm not a human, but you can refer to me based on my model name if you like! How can I assist you today?\",\n",
       " 'timestamp': 1705588697.5492659,\n",
       " 'provider': 'openai',\n",
       " 'model': 'gpt-4-1106-preview',\n",
       " 'metrics': {'input_tokens': 4,\n",
       "  'output_tokens': 53,\n",
       "  'total_tokens': 57,\n",
       "  'cost': 0.0016300000000000002,\n",
       "  'latency': 4.192759990692139,\n",
       "  'time_to_first_token': 0.624748945236206,\n",
       "  'inter_token_latency': 0.06729131824565383,\n",
       "  'tokens_per_second': 12.87934442226103},\n",
       " 'parameters': {'temperature': 1,\n",
       "  'max_tokens': 256,\n",
       "  'top_p': 1,\n",
       "  'frequency_penalty': 0,\n",
       "  'presence_penalty': 0}}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "await gpt4.async_chat(\"What's your name\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chat (stream)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Space is vast and mysterious, holding endless possibilities and uncharted territories. It is a vast expanse that stretches beyond our wildest imagination and comprehension. With billions of galaxies, each comprised of billions of stars, it is a place where wonders and mysteries reside. From the birth of stars to the formation of black holes, space has been a subject of fascination for scientists, philosophers, and dreamers alike. Moreover, space holds the potential for extraterrestrial life, as our search for habitable planets continues, fueling our curiosity and desire to unveil the secrets of the universe."
     ]
    }
   ],
   "source": [
    "response = gpt3.chat(\"Write a paragfraph about space\", is_stream=True)\n",
    "for chunk in response:\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Async version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Hi back!"
     ]
    }
   ],
   "source": [
    "async for chunk in await claude2.async_chat(\"Say hi back\", is_stream=True):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
